The invention relates to a method and an apparatus for representing a digital image on a projection surface, and the content of an image buffer being projected onto the projection surface by means of a projector.
In the context of the invention, a projection surface is in principle any desired surface. In particular a projection surface in the context of the invention can be a surface that is non-trivial in terms of geometry and photometry that, for instance, has any desired three-dimensional shape and/or any desired texture. In the context of the invention a projection surface can in particular also comprise a plurality of partial surfaces.
A system is known from DE 102 51 217 B3 for automatic calibration of multiple projector systems that has at least two projectors, one digital camera, and a control unit for controlling the projectors and the camera, automatic calibration occurring with the steps of generation, recording and image filtering of stripe patterns, finding the largest possible common projection surface, calculating the warp fields, and image warping. In this method, a plurality of video projectors are calibrated by using digital camera to record vertical and horizontal stripe patterns that are projected on a projection surface. The stripes are at a constant distance from one another that is selected depending on how complex the geometry of the projection surface is. The stripes are recorded separately for each projector, so that the number of recordings is twice the number of projectors. In addition, an image is recorded in that all of the projectors project a black image. Warping of the projection surface geometry is displayed interpolated between discrete points of a grid, the width of grid cells being determined by the spacing of the stripe patterns, which spacing cannot be reduced as desired.
Pixel-exact warping is not possible with this system. It is furthermore disadvantageous that non-white texturizing or differences in texturizing of the projection surface have a negative effect on the display.
Projecting column/line-coded patterns with phase displacements in order to acquire surface structures is known from Guehring, “Dense 3d surface acquisition by structures light using off-the shelf-components”, Proceedings of SPIE: Videometrics and Optical Methods for 3D Shape Measuring, 2001, vol. 4309, pp. 220-231.
Various algorithms and methods have been developed in order to be able to use evaluation of pixel intensities to reach an understanding regarding the definition of an image. Passive autofocus mechanisms in digital cameras use such algorithms, for instance. A series of camera images are generated at different lens settings such that these images are focused differently. Consequently, the lens setting that produces the best-focused recording is the setting whose image has the most pronounced definition. However, precise determination of definition is a function of the number of easily recognizable edges in a depiction of a scene. Recordings of natural scenes, especially in the dark, frequently do not have high contrast and therefore do not have an adequate number of edges. This has a negative effect on the quality of the passive autofocus mechanisms in the cameras.
Conventional projectors can merely be focused on exactly one image plane. If images are projected onto complex three-dimensionally structured surfaces with varying distances, they blur as the distance between the reflection image point and the focused image surface increases. This is true in particular for image details that already have less definition and that are necessary for correct depth perception of the image content. Defocused projection causes them to become even less defined, which distorts the depth perception.
In the prior art, commercially available projectors use so-called structured light in conjunction with an integrated camera to automatically determine general individual definition on flat projection screens for the entire projected image and to adapt the representation in terms of the focus to maximum definition. Similar to the passive autofocus mechanisms in cameras, light bars are projected at different lens settings. The recordings of the built-in camera are evaluated to determine the lens setting at which the definition of the recording is best.
A similar method is described, for example, in Tsai, D. M. and Chou, C. C., “A fast measure for video display inspection”, Machine Vision and Applications, Vol. 14, pp. 192-196, 2003. It is used for rapidly determining definition in CRT monitors. With this method, definition can be determined in real time. The monitor provides various binary test patterns. The definition is measured based on the assumption that the less defined the monitor is, the more the portion of bright areas increases, while the portion of dark areas decreases. The method is used on images that are recorded with a camera. The test patterns in the camera image are disassembled in a foreground and a background using the moment retaining principle. The portion of the pixels in the foreground is calculated as the definition.
Such a method merely provides an overall measurement of definition for the entire image. If such a method is used for projectors in order to set a maximum definition by focusing, on an uneven projection surface portions of the image that are outside of the focused image plane will be displayed undefined.
In order to circumvent this problem, planetariums and other virtual reality representations use laser projectors that pass over the projection surface by line and column, instead of conventional lamp projectors, in order to represent defined images on curved projection screens, for instance in domes or cylindrical rooms. Such projectors have great depth of field. They permit defined projections themselves on surfaces that are very geometrically complex when laser projectors are used without lens optics. Such a method is described in Biehling, W., Deter, C., Dube, S., Hill, B., Helling, S., Isakovic, K., Klose, S., and Schiewe, K., in “LaserCave—Some Building Blocks for Immersive Screens”, Proc. of Int. Status Conference on Virtual- and Augmented Reality, Leipzig, 2004. It presents a projection method with a combined projection surface, which method compensates the overlapping areas of the images from different projectors such that the projection surface is uniformly illuminated. The system can compensate simple uneven geometries on projection surfaces. Projection onto domed surfaces, for instance, is possible with this method.
However, this solution is extremely costly. The price for a single laser projector is currently 500 to 700 times that of a conventional digital light projector. In addition, the laser projectors used have a major disadvantage. The comprise components that generate laser beams for the RGB color channel and scanner-based projection components that use movable mirrors to deflect the laser beam. Mobile use of laser projectors is therefore very complex.
A few image reconstruction methods are known that combine photography having virtual high depth of field from a plurality of registered recordings with various focused image segments. One of these for instance is Eltoukhy, H. A. and Kavusi, S., “A Computationally Efficient Algorithm for Multi-Focus Image Reconstruction”, Proc. of SPIE Electronic Imaging, 2003. It is described therein that the calculation of the absolute gradients of two adjacent pixels is adequate for determining definition. In addition, it is indicated that decreasing the mean intensity of the input image can affect the determination of definition, and thus image reconstruction, by defocusing. It is suggested that the intensity values of the input image be normalized prior to the calculation. The image is reconstructed using a binary decision based on the assumption that a defined recording generates higher gradients than a less defined recording. Thus, an image that is defined everywhere is reconstructed in that the pixel intensity is selected from the input image that has the higher gradient. The definition is determined not only from the gradient of the pixel, but from all gradients of the pixel in a small search window. In this manner measurement fluctuations due to noise can be taken into account. Also it is taken into account that the low definition of a pixel affects the intensities of the adjacent pixel and thus also the reconstruction itself. It is suggested to smooth a binary mask that is created from definitions for decision-making in order to obtain soft transitions between partial images. The smoothed mask thus does not have any affect on the definition quality in the reconstruction.
These methods are not suitable for representing moving contents with active display devices such as projectors.
Also known in the prior art are efforts to intentionally attain artificial lack of definition on projection surfaces in order, for example, to improve the depth perception of projected image contents. Thus the publication by Majumder, A. and Welch, G., “Computer Graphics Optique: Optical Superposition of Projected Computer Graphics”, Proc. of Eurographics Workshop on Virtual Environment/Immersive Projection Technology, 2001, describes a method for producing graphic low definition effects by means of overlaid projections from two projectors whose images overlap one another completely but that are focused differently so that one projector produces a defined image while the other simultaneously produces an image with low definition.
The underlying object of the invention is to provide a method and an apparatus with which digital images can be projected onto a background with any desired surface structure and/or color such that the effect on the image caused by the background is compensated in a pixel-exact manner, at least for one special observer perspective. In one special embodiment it should be possible to attain high image definition.